Add Comprehensive Metrics Collection and Performance Reporting

Overview

This PR adds comprehensive metrics collection and performance reporting capabilities to Venom, enabling users to collect HTTP performance metrics during test execution and generate interactive HTML reports with threshold validation.

Key Features

🎯 Metrics Collection

• Opt-in metrics collection during venom run with --metrics-enabled flag
• HTTP request metrics: Response times (avg, p50, p90, p95, p99), status codes, error rates
• Test assertion metrics: Pass/fail rates for individual test checks
• k6-compatible JSON output for integration with existing tooling

🛣️ Dynamic Path Normalization (DPN)

• Endpoint aggregation: Similar URLs (e.g., /users/123, /users/456) → /users/*
• Global state management: Consistent normalization across test runs

📊 Interactive HTML Reports

• Rich visualizations: Chart.js-powered charts for performance trends, distributions, and comparisons
• Filtering & sorting: Min sample size filter, endpoint search, sortable tables
• SLA compliance: Visual threshold breach detection with detailed violator lists
• Responsive design: Bootstrap 5 with mobile-friendly layout
• Progressive enhancement: Fallback charts when Chart.js unavailable

⚡ Performance Thresholds

• YAML configuration: Flexible threshold definitions with groups and endpoint overrides
• SLA validation: Configurable min samples, tolerance percentages, and soft fail mode
• CI integration: JUnit XML output for threshold violations
• Default behavior: Soft fail (non-blocking) with --fail-on-breaches for explicit failures

Usage

Enable Metrics Collection

# Basic usage
venom run --metrics-enabled --metrics-output=metrics.json tests/

# Parallel execution
venom run --metrics-enabled --metrics-output=parallel_output/metrics_{#}.json tests/ --parallel

Generate Reports

# HTML report
venom metrics-report metrics.json --html-output=report.html

# Threshold validation
venom metrics-report metrics.json --check-thresholds --thresholds=thresholds.yml --html-output=report.html

Configure Thresholds

# thresholds.yml
options:
  min_samples: 20
  tolerance_percent: 10
  soft_fail: false

defaults:
  p95: 500ms
  p99: 1000ms
  avg: 200ms

groups:
  "POST_config/*":
    p95: 30000ms

endpoints:
  "POST_config_createtenant":
    p95: 30000ms
    avg: 25000ms

Technical Implementation

New CLI Commands

• venom run --metrics-enabled --metrics-output=FILE: Collect metrics during test execution
• venom metrics-report METRICS_FILE [OPTIONS]: Generate reports and validate thresholds

New Packages

• reporting/: Core metrics collection and threshold validation
• reporting/aggregator/: Metrics aggregation utilities
• executors/http/dpn.go: Dynamic path normalization

Integration Points

• HTTP Executor: Collects request metrics with DPN normalization
• Test Execution: Records assertion results and step outcomes
• Context Management: Metrics collector available throughout test execution

Benefits

1. Performance Visibility: Understand API performance characteristics and identify bottlenecks
2. SLA Monitoring: Automated threshold validation with configurable tolerances
3. CI/CD Integration: Non-blocking performance checks with JUnit XML output
4. Developer Experience: Interactive HTML reports with filtering and visualization

Backward Compatibility

• Fully opt-in: No impact on existing workflows without --metrics-enabled

Testing

• Comprehensive test coverage for metrics collection, DPN normalization, and threshold validation
• Integration tests for CLI commands and HTML report generation
• Validation of k6-compatible JSON output format

Ready for beta testing - All core functionality implemented and tested.